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SOFTWARE, SYSTEMS AND METHODS FOR 
MANAGING A DISTRIBUTED NETWORK 

FIELD OF THE INVENTION 

The present invention relates generally to distributed 
network systems, and more particularly to software, systems 
and methods for managing resources of a distributed net- 
work. 

BACKGROUND 

Both public and private networks have shifted toward a 
predominantly distributed computing model, and have 
grown steadily in size, power and complexity. This growth 
has been accompanied by a corresponding increase in 
demands placed on information technology to increase 
enterprise-level productivity, operations and customer/user 
support To achieve interoperability in increasingly complex 
network systems, TCP/IP and other standardized communi- 
cation protocols have been aggressively deployed. Although 
many of these protocols have been effective at achieving 
interoperability, their widespread deployment has not been 
accompanied by a correspondingly aggressive development 
of management solutions for networks using these protocols. 

Indeed, conventional computer networks provide little in 
the way of solutions for managing network resources, and 
instead typically provide what is known as "best efforts"- 
service to all network traffic. Best efforts is the default 
behavior of TCP/IP networks, in which network nodes 
simply drop packets indiscriminately when faced with 
excessive network congestion. With best efforts service, no 
mechanism is provided to avoid the congestion that leads to 
dropped packets, and network traffic is not categorized to 
ensure reliable delivery of more important data. Also, users 
are not provided with information about network conditions 
or underperforming resources. This lack of management 
frequently results in repeated, unsuccessful network 
requests, user frustration and diminished productivity. 

Problems associated with managing network resources 
are intensified by the dramatic increase in the demand for 
these resources. New applications for use in distributed 
networking environments are being developed at a rapid 
pace. These applications have widely varying performance 
requirements. Multimedia applications, for example, have a 
very high sensitivity to jitter, loss and delay. By contrast, 
other types of applications can tolerate significant lapses in 
network performance. Many applications, particularly con- 
tinuous media applications, have very high bandwidth 
requirements, while others have bandwidth requirements 
that are comparatively modest. A further problem is that 
many bandwidth-intensive applications are used for recre- 
ation or other low priority tasks. 

In the absence of effective management tools, the result of 
this increased and varied competition for network resources 
is congestion, application unpredictability, user frustration 
and loss of productivity. When networks are unable to 
distinguish unimportant tasks or requests from those that are 
mission critical, network resources are often used in ways 
that are inconsistent with business objectives. Bandwidth 
may be wasted or consumed by low priority tasks. Custom- 
ers may experience unsatisfactory network performance as a 
result of internal users placing a high load on the network. 

Various solutions have been employed, with limited 
success, to address these network management problems. 
For example, to alleviate congestion, network managers 
often add more bandwidth to congested links. This solution 
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is expensive and can be temporary — network usage tends to 
shift and grow such that the provisioned link soon becomes 
congested again. This often happens where the underlying 
cause of the congestion is not addressed. Usually, it is 

5 desirable to intelligently manage existing resources, as 
opposed to "over-provisioning," i.e. simply providing more 
resources to reduce scarcity. 

A broad, conceptual class of management solutions may 
be thought of as attempts to increase "awareness" in a 

10 distributed networking environment. The concept is that 
where the network is more aware of applications or other 
tasks running on networked devices, and vice versa, then 
steps can be taken to make more efficient use of network 
resources. For example, if network management software 
becomes aware that a particular user is running a low 

15 priority application, then the software could block or limit 
that user's access to network resources. If management 
software becomes aware that the network population at a 
given instance includes a high percentage of outside 
customers, bandwidth preferences and priorities could be 

20 modified to ensure that the customers had a positive expe- 
rience with the network. In the abstract, increasing applica- 
tion and network awareness is a desirable goal, however 
appbeation vendors largely ignore these considerations and 
tend to focus not on network infrastructure, but rather on 

25 enhancing application functionality. 

Quality of service ("QoS") and policy-based management 
techniques represent efforts to bridge the gap between 
networks, applications and users in order to more efficiently 
manage the use of network resources. QoS is a term refer- 

30 ring to techniques which allow network-aware applications 
to request and receive a predictable level of service in terms 
of performance specifications such as bandwidth, jitter, 
delay and loss. Known QoS methods include disallowing 
certain types of packets, slowing transmission rates, estab- 

35 lishing distinct classes of services for certain types of 
packets, marking packets with a priority value, and various 
queuing methods. In a distributed environment having 
scarce resources, QoS techniques necessarily introduce 
unfairness into the system by giving preferential treatment to 

40 certain network traffic. 

Policy-based network management uses policies, or rules, 
to define how network resources are to be used. In a broad 
sense, a policy includes a condition and an action. An 
example of a policy could be to block access or disallow 

45 packets (action) if the IP source address of the data is 
included on a list of disallowed addresses (condition). One 
use of policy-based network management techniques is to 
determine when and how the unfairness introduced by QoS 
methods should apply. 

50 Policy-based management solutions typically require that 
network traffic be classified before it is acted upon. The 
classification process can occur at various levels of data 
abstraction, and may be described in terms of layered 
communication protocols that network devices use to com- 

55 municate across a network link. There are two protocol 
layering models which dominate the field. The first is the 
OSI reference model, depicted in FIG. 1. The layers of the 
OSI model are: application (layer 7), presentation (layer 6), 
session (layer 5), transport (layer 4), network (layer 3), data 

60 link (layer 2) and physical (layer 1). The second major 
model forms the basis for the TCP/IP protocol suite. Its 
layers are application, transport, network, data link and 
hardware, as also depicted in FIG. 1. The TCP/IP layers 
correspond in function to the OSI layers, but without a 

65 presentation or session layer. In both models, data is pro- 
cessed and changes form as it is sequentially passed between 
the layers. 
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Known policy based management solutions and QoS FIG. 11C is a flowchart depicting yet another method 

methods typically classify data by monitoring data flows at according to the present invention for allocating bandwidth 

the transport layer and below. For example, a common among a plurality of computers. 

multi-parameter classifier is the well known "five-tuple" nG UD is a flowchart depicting yet another method 

consisting of OP source address, IP donation address IP 5 according to mc prcscnt invention for allocating bandwidth 

protocol, TCP/UDP source port and TCP/UDP destination * ^ of ers . 

port). These parameters are all obtained at the transport and * . ^ , . . . _. 

network layers of the models. The large majority of existing nG - 12 * a flowchart depicting a method according to the 

policy-based, QoS solutions are implemented by monitoring present mvention for monitoring the status of network 

and classifying network activity at these protocol layers. resources. 

However, the higher the protocol layer, the more definitive FIG. 13 is a view of a main configuration screen of a 

and specific the available data and classifiers. Because configuration utility according to the present invention, 

conventional policy-based, QoS systems do not employ pj G u {s a ^ of anothef configuration smen of (he 

classifiers at higher than the transport layer, they cannot configuration utility. 

employ policy-based techniques or QoS methods using the . . , . « r 

richer and more detailed data available at the higher layers. ™ FIG. 15 is a view of yet another configuration screen of 

The conventional systems are thus limited in their ability to configuration utility. 

make the network more application-aware and vice versa. FIG. 16 is a view of yet another configuration screen of 

In addition, the known systems for managing network the configuration utility, 
resources do not effectively address the problem of band- 
width management. Bandwidth is often consumed by low 20 DETAILED DESCRIPTION OF THE 
priority tasks at the expense of business critical applications. INVENTION 
In systems that do provide for priority based bandwidth jhe present invention provides a system and method for 
allocations, the bandwidth allocations are static and are not managing network resources in a distributed networking 
adjusted dynamically in response to changing network con- environment, such as distributed network 10 depicted in 
ditions. 25 FIG. 2. Hie invented software, system and methods increase 
BRIEF DESCRIPTION OF THE DRAWINGS productivity and customer/user satisfaction, minimize frus- 
_ . , A . . _ , r-™™ tration associated with using the network, and ultimately 
FIG. 1 is a conceptual depiction of the OSI and TCP/IP ^ network TQSQmcts are ^ in a way C0Dsislent 

layered protocol models. ^ underlying business or other objectives. 

FIG. 2 is a view of a distributed network system m which 30 ^ ^ comporjeri ts, 

the software systems and methods of the present mvention ^ ^ ^ a a]so ^ && & 

may be deployed. point: The agents and control points are deployed throughout 

nG.3isaschemaucviewofacomputmgdevicethatmay distributed network i 0 , and interact with each other to 

be included with the distributed network system of FIG. 2. accomplish the above goa i s . A plurality of agents may be 

FIG. 4 is a block diagram view depicting the relationship 35 deployed to intelligently couple clients, servers and other 

between the distributed agent modules and control modules computing devices to the underlying network. The deployed 

of the present invention, and depicting the relationship , gclUs ~™ tGr) analyze and act upon network events rclat- 

between the computing devices with which the agent and ^ t0 the netW0I fc e d devices with which they are associated, 

control modules are associated. ^ agents arc centrally coordinated and/or controlled by 

FIG. 5 is a block diagram view depicting various com- 4 one or more control points. The agents and control points 
ponents of the invented software, system and methods, interact to control and monitor network events, track opera- 
including two agent modules, a control module and a tional and congestion status of network resources, select 
configuration utility, the diagram also depicting the inter- optimum targets for network requests, dynamically manage 
connection of the components with layered communications bandwidth usage, and share information about network 
protocol software and a network link. 45 conditions with customers, users and IT personnel. 

FIG. 6 is a block diagram depicting a configuration of an M indicated, distributed network 10 may include a local 

agent module according to the present invention with a network 12 and a plurality of remote networks 14 linked by 

computing device. a puD lic network 16 such as the Internet. The local network 

FIG. 7 is a block diagram depicting another configuration 5Q an d remote networks are connected to the public network 

of an agent module according to the present invention with with network infrastructure devices such as routers 18. 

a computing device. Local network 12 will be seen to include servers 20 and 

FIG. 8 is a block diagram depicting yet another configu- cueDt devices such as client computers 22 interconnected by 

ration of an agent module according to the present invention network link 24. Additionally, local network 12 may include 

with a computing device. 55 anv number and variety of devices, including file servers, 

FIG. 9 is a block diagram depicting various component applications servers, mail servers, WWW servers, databases, 

parts of an embodiment of an agent module according to the client computers, remote access devices, storage devices, 

present invention. printers and network infrastructure devices such as routers, 

FIG. 10 is a block diagram depicting various component bridges, gateways, switches, hubs and repeaters. Remote 

parts of an embodiment of a control module according to the 60 networks 14 may similarly include any number and variety 

present invention. of networked devices. 

FIG. 11A is a flowchart depicting a method according to Indeed, virtually any type of computing device may be 

the present invention for allocating bandwidth among a connected to the networks depicted in FIG. 2, including 

plurality of computers. general purpose computers, laptop computers, handheld 

FIG. 11B is a flowchart depicting another method accord- 65 computers, wireless computing devices, mobile telephones, 

ing to the present invention for allocating bandwidth among pagers, pervasive computing devices and various other spe- 

a plurality of computers. cialty devices. Typically, many of the connected devices are 
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general purpose computers which have at least some of the OSI model has seven layers, including an application layer, 

elements shown in FIG. 3, a block diagram depiction of a a presentation layer, a session layer, a transport layer, a 

computer system 40. Computer system 40 includes a pro- network layer, a data link layer and a physical layer. The 

cesser 42 that processes digital data. The processor may be TCP/IP-based model includes an application layer, a trans- 

a complex instruction set computing (CISC) s port hyer, a network layer, a data link layer and a physical 

microprocessor, a reduced instruction set computing (RISC) j avcr 

microprocessor, a very long ir*trucdon word (VUW) ' k 

microprocessor, a processor implementing a combination of * J . 

instruction sets, a microcontroller, or virtually any other communicatee. Conceptual^, all of the protocol layers he 

processor/controller device. 1T,e processor may be a single .„ m a data transmission path that is between ^ an application 

j , ... r j • 10 program running on the particular networked device and the 

device or a plurality of devices. ♦ i r i *u r ** i u ■ i . . *u 

n n . .„ . . » « t A - network link, with the application layer being closest to the 

Referrinc still to FIG. 3, it will be noted that processor 42 f r / _ & 

ix^iiiugatm iw*ivj. iu ^ apphcaUon program. When data is transferred from an 

is coupled to a bus 44 which transmits signals between the i- , 

F , , . & . 4 apphcation program running on one computer across the 

processor and other components in the computer system. f i , i- • iL 

Si_ i -it i j * .i • . .u « .u u u network to an application program running on another 

Those skilled ,□ the art will appreciate that the bus may be J5 ^ ^ ^fj,,/^ ^ locol 

a single bus or a plurality of buses. A memory 46 is coupkd rf ^ first co ^ ^ network ^ ^ 
to bus 44 and compnses a random access memory (RAM) fa ^ ^ on ^ 
device 47 (referred to as mam memory) that stores infor- r ° r ' r 
mation or other intermediate data during execution by In both of me de P lcled models ' me application layer is 
processor 42. Memory 46 also includes a read only memory w responsible for interacting with an operating system of the 
(ROM) and/or other static storage device 48 coupled to the networked device and for providing a window for applica- 
bus that stores information and instructions for processor 42. tion programs running on the device to access the network. 
Abasic input/output system (BIOS) 49, containing the basic ^ transport layer is responsible for providing reliable, 
routines that help to transfer information between elements end-to-end data transmission between two end points on a 
of the computer system, such as during start-up, is stored in „ network, such as between a client device and a server 
ROM 48. A data storage device 50 also is coupled to bus 44 computer, or between a web server and a DNS server, 
and stores information and instructions. The data storage Depending on the particular transport protocol, transport 
device may be a hard disk drive, a floppy disk drive, a functionality may be realized using either connection- 
CD-ROM device, a flash memory device or any other mass ° nented ° r connectionless data transfer. The network layer 
storage device. In the depicted computer system, a network 30 typically is not concerned with end-to-end delivery, but 
interface 52 also is coupled to bus 44. The network interface rauler wllh forwarding and routing data to and from nodes 
operates to connect the computer system to a network (not be ™ endpoints The layers below the transport and 
shown) network layers perform other functions, with the lowest 

Computer system 40 may also include a display device lev ^ ^dressing the physical and electrical issues of trans- 
controller 54 coupled to bus 44. The display device control- 35 miUlDg f&W bltS acn * 8 * DCtW ° rk ^ . f 
ler allows coupling of a display device to the computer The present invention is applicable to a wide variety of 
system and operates to interface the display device to the network environments employing communications proto- 
computer system. The display device controller 54 may be, —Is adhering to either of the layered models depicted m 
for example, a monochrome display adapter (MDA) card, a FIG. or t0 an y other la y ered modeL Furthermore, the 
color graphics adapter (CGA) card, or other display device 40 mvention is applicable to any type of network topology, and 
controller. The display device (not shown) may be a televi- to networks using both physical and wireless connections, 
sion set, a computer monitor, a flat panel display or other The present invention provides software, systems and 
display device. The display device receives information and methods for managing the resources of an enterprise 
data from processor 42 through display device controller 54 network, such as that depicted in FIG. 2. This is accom- 
and displays the information and data to the user of com- 45 plished using two main interacting software components, an 
puter system 40. agent and a control point, both of which are adapted to run 

An input device 56, including alphanumeric and other on, or be associated with, computing devices such as the 

keys, typically is coupled to bus 44 for communicating computing device described with reference to FIG. 3. As 

information and command selections to processor 42. seen in FIG. 4, a plurality of agents 70 and one or more 

Alternatively, input device 56 is not directly coupled to bus 50 control points 72 may be deployed throughout distributed 

44, but interfaces with the computer system via infra-red network 74 by loading the agent and control point software 

coded signals transmitted from the input device to an modules on networked computing devices such as clients 22 

infra-red receiver in the computer system (not shown). The and server 20. As will be discussed in detail, the agents and 

input device may also be a remote control unit having keys control points may be adapted and configured to enforce 

that select characters or command selections on the display 55 system policies; to monitor and analyze network events, and 

device. take appropriate action based on these events; to provide 

The various computing devices coupled to the networks valuable information to users of the network; and ultimately 
of FIG. 2 typically communicate with each other across t0 ensure that network resources are efficiently used in a 
network links using communications software employing manner consistent with underlying business or other goals, 
various communications protocols. The communications 60 The invented software, systems and methods may be 
software for each networked device typically consists of a configured using a third software component, to be later 
number of protocol layers, through which data is sequen- discussed in more detail with reference to FIGS. 5 and 
tially transferred as it is exchanged across a network link 13-16. Typically, this configuration utility is a platform- 
between devices. FIG. 1 respectively depicts the OSI layered independent application that provides a graphical user inter- 
protocol model and a layered model based on the TCP/IP 65 face for centrally managing configuration information for 
suite of protocols. These two models dominate the field of the control points and agents. In addition, the configuration 
network communications software. As seen in the figure, the utility may be adapted to communicate and interface with 
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other management systems, including management plat- control points monitor the network and devices connected to 
forms supplied by other vendors. the network to determine when various rules apply and 
As indicated in FIG. 4, each control point 72 is typically whether the conditions accompanying those rules are satis- 
associated with multiple agents 70, and the associated agents fied. Once the agents and/or control points determine that 
are referred to as being within a domain 76 of the particular 5 action is required, they take the necessary actions) to 
control point. The control points coordinate and control the enforce the system policies. 

activity of the distributed agents within their domains. In For example, successful businesses often strive to provide 

addition, the control points monitor the status of network excellent customer services. This underlying business goal 

resources, and share this information with management and can be translated into many different policies defining how 

support systems and with the agents. 10 network resources are to be used. One example of such a 

Control points 72 and agents 70 may be flexibly deployed policy would be to prevent or limit access to non-business 

in a variety of configurations. For example, each agent may critical applications when performance of business critical 

be associated with a primary control point and one or more applications is degraded beyond a threshold point. Another 

backup control points that will assume primary control if example would be to use QoS techniques to provide a 

necessary. Such a configuration is illustrated in FIG. 4, 15 guaranteed or high level of service to e-commerce applica- 

where control points 72 within the dashed lines function as lions. Yet another example would be to dynamically increase 

primary connections, with the control point associated with network bandwidth allocated to a networked computer 

server device 20 functioning as a backup connection for all whenever it is accessed by a customer Also, bandwidth for 

of the depicted agents. In addition, the invented management various applications might be restricted during times when 

solution may be deployed so that one control point coordi- 20 there is heavy use of network resources by customers, 

nates and controls the activity of a single domain, or of Control points 72 would access these policies and provide 

multiple domains. Alternatively, one domain may be con- policy data to agents 70. Agents 70 and control points 72 

trolled and coordinated by the cooperative activity of mul- would communicate with each other and monitor the net- 

tiple control points. In addition, agents may be configured to work to determine how many customers were accessing the 

have embedded control point functionality, and may there- 25 network, what computers the customers) were accessing, 

fore operate without an associated control point entity. and what applications were being accessed by the custom- 

The agents monitor network resources and the activity of ers. Once the triggering conditions were detected, the agents 

the device with which they are associated, and communicate and control points would interact to re-allocate bandwidth, 

this information to the control points. In response to moni- 3Q provide specified service levels, block or restrict various 

tored network conditions and data reported by agents, the non-customer activities, etc. 

control points alter the behavior of particular agents in order Another example of policy-based management would be 

to provide the desired network services. The control points to define an optimum specification of network resources or 

and agents may be loaded on a wide variety of devices, service levels for particular types of network tasks. The 

including general purpose computers, servers, routers, hubs, 35 particular policies would direct the management entities to 

palm computers, pagers, cellular telephones, and virtually determine whether the particular task was permitted, and if 

any other networked device having a processor and memory. permitted, the management entities would interact to ensure 

Agents and control points may reside on separate devices, or that the desired level of resources was provided to accom- 

simultaneously on the same device. plish the task. If the optimum resources were not available, 

FIG. 5 illustrates an example of the way in which the 40 the applicable policies could further specify that the 

various components of the invented software, systems and requested task be blocked, and that the requesting user be 

methods may be physically interconnected with a network provided with an informative message detailing the reason 

link 90. The components are all connected to network link why the request was denied. Alternatively, the policies could 

90 by means of layered communications protocol software specify that the user be provided with various options, such 

92. The components communicate with each other via the 45 as proceeding with the requested task, but with sub-optimal 

communications software and network link. As will be resources, or wailing to perform the task until a later time, 

appreciated by those skilled in the art, network link 90 may For example, continuous media applications such as IP 

be a physical or wireless connection, or a series of links telephony have certain bandwidth requirements for optimum 

including physical and wireless segments. More specifically, performance, and are particularly sensitive to network jitter 

the depicted system includes an agent 70 associated with a 50 and delay. Policies could be written to specify a desired level 

client computing device 22, including an application pro- of service, including bandwidth requirements and threshold 

gram 98. Another agent is associated with server computing levels for jitter and delay, for client computers attempting to 

device 20. The agents monitor the activity of their associated run IP telephony applications. The policies would further 

computing devices and communicate with control point 72. direct the agents and control modules to attempt to provide 

Configuration utility 106 communicates with all of the other 55 the specified level of service. Security checking could also 

components, and with other management systems, to con- be included to ensure that the particular user or client 

figure the operation of the various components and monitor computer was permitted to run the application. In the event 

the status of the network. that the specified service level could not be provided, the 

The system policies that define how network resources are requesting user could be provided with a message indicating 
to be used may be centrally defined and tailored to most 60 lhal me resources for the request were not available. The 
efficiently achieve underlying goals. Defined policies are user could also be offered various options, including pro- 
accessed by the control points, which in turn communicate ceeding with a sub-optimal level of service, placing a 
various elements and parameters associated with the policies conventional telephone call, waiting to perform the task 
to the agents within their domain. At a very basic level, a until a later time, etc. 

policy contains rules about how network resources are to be 65 The software, system and methods of the present inven- 

used, with the rules containing conditions and actions to be tion may be used to implement a wide variety of system 

taken when the conditions are satisfied. The agents and policies. The policy rules and conditions may be based on 
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any number of parameters, including IP source address, IP 
destination address, source port, destination port, protocol, 
application identity, user identity, device identity, URL, 
available device bandwidth, application profile, server 
profile, gateway identity, router identity, time-of-day, net- 
work congestion, network load, network population, avail- 
able domain bandwidth and resource status, to name but a 
partial list. The actions taken when the policy conditions are 
satisfied can include blocking network access, adjusting 
service levels and/or bandwidth allocations for networked 
devices, blocking requests to particular URLs, diverting 
network requests away from overloaded or underperforming 
resources, redirecting network requests to alternate 
resources and gathering network statistics. 

Some of the parameters listed above may be thought of as 
"client parameters,** because they are normally evaluated by 
an agent monitoring a single networked client device. These 
include IP source address, IP destination address, source 
port, destination port, protocol, application identity, user 
identity, available device bandwidth and URL. Other 
parameters, such as application profile, server profile, gate- 
way identity, router identity, time-of-day, network 
congestion, network load, network population, available 
domain bandwidth and resource status may be though of as 
"system parameters*' because they pertain to shared 
resources, aggregate network conditions or require evalua- 
tion of data from multiple agent modules. Despite this, there 
is not a precise distinction between client parameters and 
system parameters. Certain parameters, such as time-of-day, 
may be considered either a client parameter or a system 
parameter, or both. 

Policy-based network management, QoS implementation, 
and the other functions of the agents and control points 
depend on obtaining real-time information about the net- 
work. As will be discussed, the present invention is an 
improvement over known policy-based QoS management 
solutions because of the enhanced ability to obtain detailed 
inform alien about network conditions and the activity of 
networked devices. Many of the policy parameters and 
conditions discussed above are accessible in the present 
invention due to the particular way the agent modules are 
coupled to the communications software of their associated 
devices. Also, as the above examples suggest, managing 
bandwidth and ensuring its availability for core applications 
is an increasingly important consideration in managing 
networks. The present invention provides an improved sys- 
tem and method for dynamically allocating bandwidth and 
controlling bandwidth usage in response to changing net- 
work conditions. 

The ability of the present invention to flexibly deploy 
policy-based, QoS management solutions based on detailed 
information about network conditions has a number of 
significant benefits. These benefits include reducing frustra- 
tion associated with using the network, reducing help calls 
to IT personnel, increasing productivity, lowering business 
costs associated with managing and maintaining enterprise 
networks, and increased customer/user loyalty and satisfac- 
tion. Ultimately, the invented systems and methods ensure 
that network resources are used in a way that is consistent 
with underlying goals and objectives. 

The ability of the present invention to implement policy- 
based QoS between the application and transport layers has 
another advantage. This allows support for encryption and 
other security implementations carried out using Virtual 
Private Networking (VPN) or IPSec protocol. 

Referring now to FIGS. 6-9, the agent module will be 
more particularly described. The basic functions of the agent 
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module are monitoring the status and activities of its asso- 
ciated client, server, pervasive computing device or other 
computing device, communicating this information to one or 
more control points, enforcing system policies under the 
5 direction of the control points, and providing messages to 
network users and administrators concerning network con- 
ditions. FIGS. 6-8 are conceptual depictions of networked 
computing devices, and show how the agent software is 
associated with the networked devices relative to layered 
protocol software used by the devices for network commu- 
nication. 

As seen in FIG. 6, agent 70 is interposed between appli- 
cation program 122 and a communications protocol layer for 
providing end-to-end data transmission, such as transport 

5 layer 124 of communications protocol stack 92. Typically, 
the agent modules of the present invention are used with 
network devices that employ layered communications soft- 
ware adhering to either the OSI or TCP/IP-based protocol 
models. Thus, agent 70 is depicted as "interposed," i.e. in a 

^ data path, between an application program and a transport 
protocol layer. However, it will be appreciated by those 
skilled in the art that the invented agent may be used with 
protocol software not adhering to either the OSI or TCP/IP 
models, but that nonetheless includes a protocol layer pro- 

25 viding transport functionality, i.e. providing for end-to-end 
data transmission. 

Because of the depicted position within the data path, 
agent 70 is able to monitor network traffic and obtain 
information that is not available by hooking into transport 

30 layer 124 or the layers below the transport layer. At the 
higher layers, the available data is richer and more detailed. 
Hooking into the stack at higher layers allows the network 
to become more "application -aware" than is possible when 
monitoring occurs at the transport and lower layers. 

35 The agent modules may be interposed at a variety of 
points between application program 122 and transport layer 
124. Specifically, as shown in FIGS. 7 and 8, agent 70 may 
be associated with a client compuier so thai it is adjacent an 
application programming interface (API) adapted to provide 

4 0 a standardized interface for application program 122 to 
access a local operating system (not shown) and communi- 
cations stack 92. In FIG. 7, agent 70 is adjacent a winsock 
API 128 and interposed between application program 122 
and the winsock interface. FIG. 8 shows an alternate 

45 configuration, in which agent 70 is again adjacent the 
winsock interface, but the winsock interface is interposed 
between application program 122 and agent 70. With either 
configuration, the agent is interposed between the transport 
layer 124 of communications stack 92 and is adapted to 

50 directly monitor data received by or sent from the winsock 
interface. 

As shown in FIG. 8, agent 70 may be configured to hook 
into lower layers of communications stack 92. This allows 
the agent to accurately monitor network traffic volumes by 

55 providing a correction mechanism to account for data com- 
pression or encryption occurring at protocol layers below 
transport layer 124. For example, if compression or encryp- 
tion occurs within transport layer 124, monitoring at a point 
above the transport layer would yield an inaccurate measure 

60 of the network traffic associated with the computing device. 
Hooking into lower layers with agent 70 allows network 
traffic to be accurately measured in the event that 
compression, encryption or other data processing that quali- 
tatively or quantitatively affects network traffic occurs at 

65 lower protocol layers. 

An embodiment of the agent module is depicted in FIG. 
9. As shown, agent 70 may include a redirector module 130, 
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a traffic control module 132, an administrator module 134, accumulates "credits" for intervals in which it does not 

a DNS module 136, a pop app module 138, a message broker release any waiting data. When enough credits are 

module 140, a system services module 142, and a popapp accumulated, the waiting message is released for forwarding 

144. Redircctor module 130 intercepts winsock API calls to the network. 

made by applications running on networked devices such as 5 Similarly, to control the rate at which network traffic is 
the client computers depicted in FIGS. 2 and 3. Redircctor received, traffic control module 132 may be configured to 
module 130 then hands these calls to one or more of the maintain a plurality of separate receive queues, such as 
other agent components for processing. As discussed with receive queues 132£. In addition to the methods discussed 
reference to FIGS. 6-8, redirector module is positioned to above, various other methods may be employed to control 
allow the agent to monitor data at a data transmission point 10 the rate at which network traffic is sent and received by the 
between an application program running on the device and queues. Also, the behavior of the transmit and receive 
the transport layer of the communications stack. Depending queues may be controlled through various methods to con- 
on the configuration of the agent and control point, the trol jitter, delay, loss and response time for network con- 
intercepted winsock calls may be rejected, changed, or nections. 

passed on by agent 70. 15 ^ transm i t anc j receive queues may also be configured 

Traffic control module 132 implements QoS and system to detect network conditions such as congestion and slow 

policies and assists in monitoring network conditions. Traf- responding applications or servers. For example, for each 

fic control module 132 implements QoS methods by con- application, transmitted packets or other data units may be 

trolling the network traffic flow between applications run- timestamped when passed out of a transmit queue. When 

ning on the agent device and the network link. The traffic 20 corresponding packets are received for a particular 

flow is controlled to deliver a specified network service application, the receive and send times may be compared to 

level, which may include specifications of bandwidth, data detect network congestion and/or slow response times for 

throughput, jitter, delay and data loss. various target resources. This information may be reported 

To provide the specified network service level, traffic to the control points and shared with other agents within the 

control module 132 may maintain a queue or plurality of 25 domain. The response time and other performance informa- 

queues. When data is sent from the client to the network, or lion obtained by comparing transmit and receive times may 

from the network to the client, redirector module 130 also be used to compile and maintain statistics regarding 

intercepts the data, and traffic module 132 places the indi- various network resources. 

vidual units of data in the appropriate queue. The control ^ Using this detection and reporting mechanism, a control 

points may be configured to periodically provide traffic point may be configured to reduce network loads by instruct- 

control commands, which may include the QoS parameters ing traffic control module 132 to close low priority sessions 

and- service specifications-discussed- above—In response, —and block additional sessions whenever heavy network 

traffic control module 132 controls the passing of data into, congestion is reported by one of the agents. In conjunction, 

through or out of the queues in order to provide the specified ^ as will be explained, popapp 138 module may provide a 

service level. message to the user explaining why sessions are being 

More specifically, the outgoing traffic rate may be con- closed. In addition to closing the existing sessions, the 
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queues, such as transmission queues 132a. When an appli- block any further sessions. This action may also be accom- 

cation or process is invoked by a computing device with 4Q panied by a user message in response to attempts to launch 

which agent 70 is associated, a priority level is assigned to a new application or network process. When the network 

the application, based on centrally defined policies and load is reduced, the control point will send a message to the 

priority data supplied by the control point. Specifically, as agents allowing sessions. 

will be discussed, the control points maintain user profiles, In addition to identifying congestion and slow response 

applications profiles and network resource profiles. These 45 times, traffic control module 132 may be more generally 

profiles include priority data which is provided to the agents. configured to aid in identifying downed or under-performing 

Transmission queues 132a may be configured to release network resources. When a connection to a target resource 

data for transmission to the network at regular intervals. fails, traffic module 132 notifies popapp modules 138, which 

Using the parameters specified in traffic control commands in turn launches an executable to perform a root-cause 

issued by a control point, traffic module 132 calculates how 50 analysis of the problem. Agent 70 then provides the control 

much data can be released from the transmission queues in point with a message identifying the resource and its status, 

a particular interval. For example, if the specified average if possible. 

traffic rate is 100 KBps and the queue release interval is 1 In addition, when a connection fails, popapp module 138 

ms, then the total amount of data that the queues can release may be configured to provide a message to the user, includ- 

in a given interval is 100 bits. The relative priorities of the 55 ing an option to initiate an autoconnect routine targeting the 

queues containing data to be transmitted determine how unavailable resource. Enabling autoconnect causes the agent 

much of the allotment may be released by each individual to periodically retry the unavailable resource. This feature 

queue. For example, assuming there are only two queues, Ql may be disabled, if desired, to allow the control point to 

and Q2, that have data queued for transmission, Ql will be assume responsibility for determining when the resource 

permitted to transmit 66.66% of the overall allotted interval 60 becomes available again. As will be later discussed, the 

release if its priority is twice that of Q2. Q2 would only be invented system may be configured so that the control 

permitted to release 33.33% of the allotment. If their pri- modules assume responsibility for monitoring unavailable 

orities were equal, each queue would be permitted to release resources in order to minimize unnecessary network traffic. 

50% of the interval allotment for forwarding to the network As discussed below, various agent components also mooi- 

link. 65 tor network conditions and resource usage for the purpose of 

If waiting data is packaged into units that are larger than compiling statistics. An additional function of traffic control 

the amount a given queue is permitted to release, the queue module 132 is to aid in performing these functions by 
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providing io formation lo other agent components regarding 
accessed resources, including resource performance and 
frequency of access. 

As suggested in the above discussion of traffic control 
module 132, popapp module 138 stores and is responsible 5 
for launching a variety of small application modules such as 
application 144, known as popapps, to perform various 
operations and enhance the functioning of the invented 
system. Popapps detect and diagnose network conditions 
such as downed resources, provide specific messages to 10 
users and IT personnel regarding errors and network 
conditions, and interface with other information 
management, reporting or operational support systems, such 
as policy managers, service level managers, and network 
and system management platforms. Popapps may be cus- 15 
tomized to add features to existing products, to tailor prod- 
ucts for specific customer needs, and to integrate the 
invented software, systems and methods with technology 
supplied by other vendors. 

Administrator module 134 interacts with various other 20 
agent modules, maintains and provides network statistics, 
and provides an interface for centrally configuring agents 
and other components of the invented system. With regard 
to agent configuration, administrator module 134 interfaces 
with configuration utility 106 (shown in FIGS. 5 and 15-16), 25 
in order to configure various agent parameters. Administra- 
tor module 134 also serves as a repository for local reporting 
and statistics information to be communicated to the control 
points. Based on information obtained by other agent 
modules, administrator module 134 maintains local infor- 30 
mation regarding accessed servers, DNS servers, gateways, 
routers, switches, applications and other resources. -This 
information is communicated on request to the control point, 
and may be used for network planning or to dynamically 
alter the behavior of agents. In addition, administrator 35 
module 134 stores system policies and/or components of 
policies, and provides policy data to various agent compo- 
nents as needed to implement and enforce the policies. 
Administrator module 134 also includes support for inter- 
facing the invented system with standardized network man- 40 
agement protocols and platforms. 

DNS module 136 provides the agent with configurable 
address resolving services. DNS module 136 may include a 
local cache of DNS information, and may be configured to 45 
first resolve address requests using this local cache. If the 
request cannot be resolved locally, the request is submitted 
to a control point, which resolves the address with its own 
cache, provided the address is in the control point cache and 
the user has permission to access the address. If the request 5Q 
cannot be resolved with the control point cache, the con- 
nected control point submits the request to a DNS server for 
resolution. If the address is still not resolved at this point, the 
control point sends a message to the agent, and the agent 
then submits the request directly to its own DNS server for 55 
resolution. 

DNS module 136 also monitors address requests and 
shares the content of the requests with administrator module 
134. The requests are locally compiled and ultimately pro- 
vided to the control points, which maintain dynamically 60 
updated lists of the most popular DNS servers. In addition, 
DNS module 136 is adapted to interact with control point 72 
in order to redirect address resolving requests and other 
network requests to alternate targets, if necessary. 

Message broker module 140 creates and maintains con- 65 
nections to the one or more control points with which the 
agent interacts. The various agent components use the 
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message broker module to communicate with each other and 
with a connected control point. Message broker module 140 
includes message creator and message dispatcher processes 
for creating and sending messages to the control points. The 
message creator process includes member functions, which 
create control point messages by receiving message contents 
as parameters and encoding the contents in a standard 
network format. The message creator process also includes 
a member function to decode the messages received from 
the control point and return the contents in a format usable 
by the various components of the agent. 

After encoding by the creator process, control point 
messages are added to a transmission queue and extracted by 
the message dispatcher function for transmission to the 
control point. Messages extracted from the queue are sent to 
the agent's active control point. In addition, the dispatcher 
may be configured to ensure delivery of the message using 
a sequence numbering scheme or other error detection and 
recovery methods. 

Messages and communications from an agent to a control 
point are made using a unicast addressing mechanism. 
Communications from the control point to an agent or agents 
may be made using unicast or a multicast addressing 
scheme. When configured for multicast operation, the con- 
trol point and agents may be set to revert to unicast to allow 
for communication with devices that do not support IP 
multicast. 

Once a connection with a control point is established, 
message broker module 140 monitors the status of the 
connection and switches over to a backup control point upon 
detecting a connection failure. If both the active and backup 
connections are not active, network"traffic is passed on 
transparently. 

System services module 142 provides various support 
functions to the other agent components. First, system 
services module maintains dynamic lists of user profiles, 
server profiles, DNS server profiles, coiiUol point connec- 
tions and other data. The system services module also 
provides a tracing capability for debugging, and timer ser- 
vices for use by other agent components. System services 
module may also be configured with a library of APIs to 
interface the agent with the operating systems and other 
components of the device that the agent is associated with. 

Referring now to FIGS. 10-12, control point 72 and its 
functions will be more particularly described. As seen in 
FIG. 10, control point may include a traffic module 160, a 
server profile module 162, a DNS server profile module 164, 
a gateway profile module 166, an administrator module 168, 
a message broker module 170, a popapp interface 172 and 
a popapp 174. 

Control point traffic module 160 implements policy- 
based, QoS techniques by coordinating the service-level 
enforcement activities of the agents. As part of this function, 
traffic module 160 dynamically allocates bandwidth among 
the agents in its domain by regularly obtaining allocation 
data from the agents, calculating bandwidth allocations for 
each agent based on this data, and communicating the 
calculated allocations to the agents for enforcement. For 
example, control point 72 can be configured to recalculate 
bandwidth allocations every five seconds. During each 
cycle, between re-allocation, the agents restrict bandwidth 
usage by their associated devices to the allocated amount 
and monitor the amount of bandwidth actually used. At the 
end of the cycle, each agent reports the bandwidth usage and 
other allocation data to the control point to be used in 
re-allocating bandwidth. 
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During re-allocation, traffic module 160 divides the total 
bandwidth available for the upcoming cycle among the 
agents within the domain according to the priority data 
reported by the agents. The result is a configured bandwidth 
CB particular to each individual agent, corresponding to that 5 
agent's fair share of the available bandwidth. The priorities 
and configured bandwidths are a function of system policies, 
and may be based on a wide variety of parameters, including 
application identity, user identity, device identity, source 
address, destination address, source port, destination port, 10 
protocol, URL, time of day, network load, network 
population, and virtually any other parameter concerning 
network resources that can be communicated to, or obtained 
by the control point. The detail and specificity of client-side 
parameters that may be supplied to the control point is 15 
greatly enhanced in the present invention by the position of 
agent redirector module 130 relative to the layered commu- 
nications protocol stack. The high position within the stack 
allows bandwidth allocation and, more generally, policy 
implementation, to be performed based on very specific ^ 
triggering criteria. This greatly enhances the flexibility and 
power of the invented software, systems and methods. 

The priority data reported by the agents may include 
priority data associated with multiple application programs 
running on a single networked device. In such a situation, 25 
the associated agent may be configured to report an "effec- 
tive application priority," which is a function of the indi- 
vidual application priorities. For example, if device A were 
running two application programs and device B were run- 
ning a single application program, device A's effective 30 
application priority would be twice that of device B, assum- 
ing that the individual priorities of all three applications 
were the same. The reported priority data for a device 
running multiple application programs may be further 
refined by weighting the reported priority based on the 35 
relative degree of activity for each application program. 
Thus, in the previous example, if one of the applications 
running on device A was dormant or idle, the contribution of 
that application to the effective priority of device A would be 
discounted such that, in the end, device A and device B 40 
would have nearly the same effective priority. To determine 
effective application priority using this weighted method, 
the relative degree of activity for an application may be 
measured in terms of bandwidth usage, transmitted packets, 
or any other activity-indicating criteria. 45 

In addition to priority data, each agent may be configured 
to report the amount of bandwidth UB used by its associated 
device during the prior period, as discussed above. Data is 
also available for each device's allocated bandwidth AB for 
the previous cycle. Traffic module 160 may compare con- 50 
figured bandwidth CB, allocated bandwidth AB or utilized 
bandwidth UB for each device, or any combination of those 
three parameters to determine the allocations for the upcom- 
ing cycle. To summarize the three parameters, UB is the 
amount the networked device used in the prior cycle, AB is 55 
the maximum amount they were allowed to use, and CB 
specifies the device's "fair share" of available bandwidth for 
the upcoming cycle. 

Both utilized bandwidth UB and allocated bandwidth AB 
may be greater than, equal to, or less than configured 60 
bandwidth CB. This may happen, for example, when there 
are a number of networked devices using less than their 
configured share CB. To efficiently utilize the available 
bandwidth, these unused amounts are allocated to devices 
requesting additional bandwidth, with the result being that 65 
some devices are allocated amount AB that exceeds their 
configured fair share CB. Though AB and UB may exceed 



CB, utilized bandwidth UB cannot normally exceed allo- 
cated bandwidth AB, because the agent traffic control mod- 
ule enforces the allocation. 

Any number of processing algorithms may be used to 
compare CB, AB and UB for each agent in order to calculate 
a new allocation, however there are some general principles 
which are often employed. For example, when bandwidth is 
taken away from devices, it is often desirable to first reduce 
allocations for devices that will be least affected by the 
downward adjustment. Thus, traffic module 160 may be 
configured to first reduce allocations of clients or other 
devices where the associated agent reports bandwidth usage 
UB below the allocated amount AB. Presumably, these 
devices won't be affected if their allocation is reduced. 
Generally, traffic module 160 should not reduce any other 
allocation until all the unused allocations, or portions of 
allocations, have been reduced. The traffic module may be 
configured to then reduce allocations that are particularly 
high, or make adjustments according to some other criteria. 

Traffic module 160 may also be configured so that when 
bandwidth becomes available, the newly-available band- 
width is provisioned according to generalized preferences. 
For example, the traffic module can be configured to provide 
surplus bandwidth first to agents that have low allocations 
and that are requesting additional bandwidth. After these 
requests are satisfied, surplus bandwidth may be apportioned 
according to priorities or other criteria. 

FIGS. 11A, 11B, 11C and 11D depict examples of various 
methods that may be implemented by traffic module 160 to 
dynamically allocate bandwidth. FIG. 11 A depicts a process 
by which traffic module 160 determines whether any adjust- 
ments to bandwidth allocations AB are necessary. Allocated 
bandwidths AB for certain agents are adjusted in at least the 
following circumstances. First, as seen in steps S4 and S10, 
certain allocated bandwidths AB are modified if the sum of 
all the allocated bandwidths AB total exceeds the sum of the 
configured bandwidths CB ntal . This situation may occur 
where, for some reason, a certain portion of the total 
bandwidth available to the agents in a previous cycle 
becomes unavailable, perhaps because it has been reserved 
for another purpose. In such a circumstance, it is important 
to reduce certain allocations AB to prevent the total alloca- 
tions from exceeding the total bandwidth available during 
the upcoming cycle. 

Second, if there are any agents for which AB<CB and 
UB«AB, the allocation for those agents is modified, as seen 
in steps S6 and S10. The allocations for any such agent are 
typically increased. In this situation, an agent has an allo- 
cation AB that is less than their configured bandwidth CB, 
i.e. their existing allocation is less than their fair share of the 
bandwidth that will be available in the upcoming cycle. 
Also, the reported usage UB for the prior cycle is at or near 
the enforced allocation AB, and it can thus be assumed that 
more bandwidth would be consumed by the associated 
device if its allocation AB were increased. 

Third, if there are any agents reporting bandwidth usage 
UB that is less than their allocation AB, as determined at step 
S8, then the allocation AB for such an agent is reduced for 
the upcoming period to free up the unused bandwidth. Steps 
S4, S6 and S8 may be performed in any suitable order. 
Collectively, these three steps ensure that certain bandwidth 
allocations are modified, i.e. increased or reduced, if one or 
more of the following three conditions are true: (1) 
AB (ow >>CB <WB/ , (2) AB<CB and UBaAB for any agent, or 
(3) UB<AB for any agent. If none of these are true, the 
allocations AB from the prior period are not adjusted. Traffic 
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module 160 modifies allocations AB as necessary at step 
S10. After all necessary modifications are made, the control 
point communicates the new allocations to the agents for 
enforcement during the upcoming cycle. 

FIG. 11B depicts re-allocation of bandwidth to ensure that 
total allocations AB do not exceed the total bandwidth 
available for the upcoming cycle. At step S18, traffic module 
160 has determined that the sum of allocations AB from the 
prior period exceed the available bandwidth for the upcom- 
ing period, i.e. AB fora/ >CB (ofB/ . In this situation, certain 
allocations AB must be reduced. As seen in steps S20 and 
S22, traffic module 160 may be configured to first reduce 
allocations of agents that report bandwidth usage levels 
below their allocated amounts, Le. UB<AB for a particular 
agent. These agents are not using a portion of their 
allocations, and thus are unaffected or only minimally 
affected when the unused portion of the allocation is 
removed. At step S20, the traffic module first determines 
whether there are any such agents. At step S22, the alloca- 
tions AB for some or all of these agents are reduced. These 
reductions may be gradual, or the entire unused portion of 
the allocation may be removed at once. 

After any and all unused allocation portions have been 
removed, it is possible that further reductions may be 
required to appropriately reduce the overall allocations 
AB^,. As seen in step S24, further reductions are taken 
from agents with existing allocations AB that are greater 
than configured bandwidth CB, i.e. AB>CB. In contrast to 
step S22, where allocations were reduced due to unused 
bandwidth, bandwidth is removed at step S24 from devices 
with existing allocations that exceed the calculated "fair 
share" for the upcoming cycle. As seen at step S26, the 
reductions taken at steps S22 and S24 may be performed 
until the total allocations AB WM/ are less than or equal to the 
total available bandwidth CB^, for the upcoming cycle. 

FIG. 11C depicts a method for increasing the allocation of 
certain agents. As discussed with reference to FIG. 11 A, 
where AB<CB and UBeAB for any agent, the allocation AB 
for such an agent should be increased. The existence of this 
circumstance has been determined at step S40. To provide 
these agents with additional bandwidth, the allocations for 
certain other agents typically need to be reduced. Similar to 
steps S20 and S22 of FIG. 11B, unutilized bandwidth is first 
identified and removed (steps S42 and S44). Again, the 
control point may be configured to vary the rate at which 
unused allocation portions are removed. If reported data 
does not reflect unutilized bandwidth, traffic module 160 
may be configured to then reduce allocations for agents 
having an allocation AB higher than their respective con- 
figured share CB, as seen in step S46. The bandwidth 
recovered in steps S44 and S46 is then provided to the agents 
requesting additional bandwidth. Any number of methods 
may be used to provision the recovered bandwidth. For 
example, preference may be given to agents reporting the 
largest discrepancy between their allocation AB and their 
configured share CB. Alternatively, preferences may be 
based on application identity, user identity, priority data, 
other client or system parameters, or any other suitable 
criteria. 

FIG. 11D depicts a general method for reallocating 
unused bandwidth. At step S60, it has been determined that 
certain allocations AB are not being fully used by the 
respective agents, i.e. UB<AB for at least one agent. At step 
S62, the allocations AB for these agents are reduced. As with 
the reductions and modifications described with reference to 
FIGS. 11A, 11B and 11 C, the rate of the adjustment may be 
varied through configuration changes to the control point. 



1,724 Bl 

18 

For example, it may be desired that only a fraction of unused 
bandwidth be removed during a single reallocation cycle. 
Alternatively, the entire unused portion may be removed and 
reallocated during the reallocation cycle. 

5 In step S64 of FIG. 11D, the recovered amounts are 
provisioned as necessary. The recovered bandwidth may be 
used to eliminate a discrepancy between the total allocations 
AB, ow/ and the available bandwidth, as in FIG. 11B, or to 
increase allocations of agents who are requesting additional 

10 bandwidth and have relatively low allocations, as in FIG. 
11C. In addition, if there is enough bandwidth recovered, 
allocations may be increased for agents requesting addi- 
tional bandwidth, i.e. UBaAB, even where the current 
allocation AB for such an agent is fairly high, e.g. AB>CB. 

15 As with the methods depicted in FIGS. 11B and 11C, the 
recovered bandwidth may be reallocated using a variety of 
methods and according to any suitable criteria. 

As indicated, traffic module 160 can be configured to vary 
the rate at which the above allocation adjustments are made. 

20 For example, assume that a particular device is allocated 64 
KBps (AB) and reports usage during the prior cycle of 62 
KBps(UB). Traffic module 160 cannot determine how much 
additional bandwidth the device would use. Thus, if the 
allocation were dramatically increased, say doubled, it is 

25 possible that a significant portion of the increase would go 
unused. However, because the device is using an amount 
roughly equal to the enforced allocation AB, it can be 
assumed that the device would use more if the allocation 
were increased. Thus, it is often preferable to provide small, 

30 incremental increases. The amount of these incremental 
adjustments and the rate at which they are made may be 
configured with the configuration utility, as will be discussed 
with reference to FIG. 16. If the device consumes the 
additional amounts, successive increases can be provided if 

3S additional bandwidth is available. 

In addition, the bandwidth allocations and calculations 
may be performed separately for the transmit and receive 
rates for the networked devices. In other words, the methods 

4Q described with reference to FIGS. 11A-11D may be used to 
calculate a transmit allocation for a particular device, as well 
as a separate receive allocation. Alternatively, the calcula- 
tions may be combined to yield an overall bandwidth 
allocation. 

45 Server profile module 162, DNS server profile module 
164, gateway profile module 166 and administrator module 
168 all interact with the agents to monitor the status of 
network resources. More specifically, FIG. 12 provides an 
illustrative example of how the control points and agents 

50 may be configured to monitor the status of resources on the 
network. The monitored resource(s) may be a server, a DNS 
server, a router, gateway, switch, application, etc. At step 
S100, a resource status change has occurred. For example, 
a server has gone down, traffic through a router has dra- 

55 matically increased, a particular application is unavailable, 
or the performance of a particular gateway has degraded 
beyond a predetermined threshold specified in a system 
policy. 

At step S102, a networked device attempts to access the 
60 resource or otherwise engages in activity on the network 
involving the particular resource. If the accessing or request- 
ing device is an agent, an executable spawned by popapp 
module 138 (FIG. 9) analyzes the resource, and reports the 
identity and status of the resource to the control point 
65 connected to the agent, as indicated at step S104. Launch of 
the popapp may be triggered by connection errors, or by 
triggering criteria specified in system policies. For example, 
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system policies may include performance benchmarks for DNS module 164 also performs certain particularized 

various network resources, and may further specify that functions in addition to aiding the resource monitoring and 

popapp analysis is to be performed when resource perfor- tracking described with reference to FIG. 12. Specifically, 

mance deviates by a certain amount from the established ih c DNS module maintains a local DNS cache for efficient 

benchmark. In addition, the control points may similarly be 5 i oca i address resolution. As discussed with reference to 

configured to launch popapps to analyze network resources. agcnt D ns module 136, the agents and control points 

Once the control point obtains the status information, the interact to resolve address requests, and may be configured 

control point reports the information to all of the agents in t0 rcso iv C addresses by first referencing local DNS data 

its domain, and instructs the agents how to handle further maintained by the agents and/or control points. Similar to 

client requests involving the resource, as indicated at steps 10 scrvcr profilc momJ i c 162, DNS module 164 also maintains 

S108 and S110. In the event that the target resource is down, statistics for use in network planning and dynamic system 

underperforming or otherwise unavailable, the instructions policies 

given to the agents will depend on whether an alternate , n addjUon (o ^ afa ^i^tra- 

resource is available. TTk control point stores dynarmcaUy modu|e 16g ^ confi tion . 

updated lists of alternate availab e resource* If an alternate 15 eleK and ^ ^ me ag ents wjthto me 

resource .s available the .nstructions provided to the agent domiin [q ^ DNS and gateway modules, 

may include an instruction to transparently redirect Uie administrator module 168 also aids m collecli and main . 

request to an alternate resource, as shown in step S108. For data concemin netwofk rcsourccs In 

example, if the control point knows of a server that m.rrors ^ ^^t^, module 168 retrieves u data 

the data of another server that has gone down, client requests « n - V , . j * »u 1* 

" ^ 6 , M 20 from centralized policy repositories, and stores the policy 

to the down server can simply be redirected to the mirror , , , , / t f t . . , ,„ 

. . . ,f , • -1 t_i , L data locally for use by the control points and agents in 

server. Alternatively, if no alternate resource is available, the - . J . r r 

Jt ., . \. entorcine system policies, 
agent can be instructed to provide a user message in the 0 , 
event of an access attempt, as seen in step S110. The Control pomt 72 also mcludes a synchronizaUon interface 
messaging function is handled by agent popapp module 138. „ ( not for synchronizing information among multiple 
In addition, popapp functionality may be employed by the contro1 P omts ™ {h ™ the domam - 
control point to report status information to other control Message broker module 170 performs various functions 
points and management platforms supplied by other ven- to enable the control point to communicate with the agents, 
dors. In addition, messages concerning resource status or Similar to agent message broker module 140, message 
network conditions may be provided via email or paging to 30 broker module 170 includes message creator and message 
IT personnel. dispatcher processes. The message creator process includes 
Still referring to FIG. 12, the control point may be member functions that receive message contents as param- 
configured to assume responsibility for tracking the status of eters and return encoded messages for ^ansmission to agents 
the resource in order to determine when it again becomes and other network entities. Functions for decoding received 
available, as shown in step S112. A slow polling technique 35 messages are also included with the message creator pro- 
is used to minimize unnecessary traffic on the network. cess. The dispatcher process transmits messages and ensures 
During the interval in which the resource is unavailable, the ^^^^^^^^ mechanism and error delec * 
agents eiiher redirect requests to the resources or provide " on arid recovery methods. 

error messages, based on the instructions provided by the Referring now to FIGS. 13-16, both the agents and 
control point, as shown in step S116. Once the control point 40 control points may be configured using configuration utility 
determines that the resource is again available, the control 106. Typically, configuration utility 106 is a platform- 
point shares this information with the agents and disables the independent application that provides a graphical user inter- 
instructions provided in steps S108 and S110, as shown in face for centrally managing configuration information for 
step SU8. the control points and agents. To configure the control points 

This method of tracking and monitoring resource status 45 and a S cnts > the configuration utility interface with adminis- 

has important advantages. First, it reduces unnecessary and trator module 134 of agent 70 and with administrator 

frustrating access attempts to unavailable resources. Instead module 168 of control point 72. Alternatively, configuration 

of repeatedly attempting to perform a task, a user's requests utility 106 may interface with administrator module 168 of 

are redirected so that the request can be serviced control point 72, and the control point in turn may interface 

successfully, or the user is provided with information about 50 ^ administrator module 134 of agent 70. 

why the attempts) was unsuccessful. With this information FIG. 13 depicts a main configuration screen 188 of 

in hand, the user is less likely to generate wasteful network configuration utility 106. As indicated, main configuration 

traffic with repeated access attempts in a short period of screen 188 can be used to view various objects managed by 

time. In addition, network traffic is also reduced by having the invented software, systems and methods, including 

only one entity, usually a control point, assume responsibil- 55 users, applications, control points, agents and other network 

ity for monitoring a resource that is unavailable. entities and resources. For example, screen frame 190 on the 

In addition to assisting these resource monitoring left side of the main configuration screen 188 may be used 

functions, server profile module 162 maintains a dynami- t0 P rese nt an expandable representation of the control points 

cally updated list of the servers accessed by agents within its that are configured for the network, 

domain. The server statistics may be retrieved using the 60 When a particular control point is selected in the main 

configuration utility, or with a variety of other existing configuration screen 188, various settings for the control 

management platforms. The server statistics may be used for point may be configured. For example, the name of the 

network planning, or may be implemented into various control point may be edited, agents and other entities may be 

system policies for dynamic enforcement by the agents and added to the control point's domain, and the control point 

control points. For example, the control points and agents 65 may be designated as a secondary connection for particular 

can be configured to divert traffic from heavily used servers agents or groups of agents. In addition, the system admin - 

or other resources. istrator may specify the total bandwidth available to agents 
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within the control point's domain for transmitting and 
receiving, as shown in FIG. 14. This bandwidth specification 
will affect the configured bandwidths CB and allocated 
bandwidths AB discussed with reference to control point 
traffic module 160 and the method depicted in FIG. 11. 5 

Configuration utility 106 also provides for configuration 
of various settings relating to users, applications and 
resources associated with a particular control point. For 
example, users may be grouped together for collective 
treatment, lists of prohibited URLs may be specified for 10 
particular users or groups of users, and priorities for appli- 
cations may be specified, as shown in FIG. 15. Priorities 
may also be assigned to users or groups of users. As 
discussed above, this priority data plays a role in determin- 
ing bandwidth allocations for the agents and their associated 1S 
devices. 

In addition, optimum and minimum performance levels 
may be established for applications or other tasks using 
network resources. Referring again to the IP telephony 
example discussed above, the configuration utility may be 2 o 
used to specify a minimum threshold performance level for 
a networked device running the IP telephony application. 
This performance level may be specified in terms of QoS 
performance parameters such as bandwidth, throughput, 
jitter, delay and loss. The agent module associated with the 2 5 
networked device would then monitor the network traffic 
associated with the IP telephony application to ensure that 
performance was above the minimum threshold. If the 
minimum level was not met, the control points and agents 
could interact to reallocate resources and provide the speci- 30 
fied minimum service level. Similarly, an optimum service 
level may be specified for various network applications and 
tasks. More generally, configuration utility 106 may be 
configured to manage system policies by providing func- 
tionality for authoring, maintaining and storing system 35 
policies, and for managing retrieval of system policies from 
other locations on a distributed network, such as a dedicated 
policy server. 

Referring now to FIG. 16, the configuration of various 
other control point and agent parameters will be discussed. 40 
As seen in the figure, configuration utility 106 may be used 
to configure the interval at which resource reallocation is 
performed. For example, the default interval for recalculat- 
ing bandwidth allocations is 5000 milliseconds, or 5 sec- 
onds. Also, as discussed above, the rate at which resource 45 
allocation occurs may be specified in order to prevent 
overcompensation, unnecessary adjustments to allocations, 
and inefficient reconfigurations. Specifically, the percentage 
of over-utilized bandwidth that is removed from a client 
device and reallocated elsewhere may be specified with the 50 
configuration utility, as seen in FIG. 16. In addition, the rate 
at which agents provide feedback to the control points 
regarding network conditions or activities of their associated 
devices may be configured. 

It is believed that the disclosure set forth above encom- 55 
passes multiple distinct inventions with independent utility. 
While each of these inventions has been disclosed in its 
preferred form, the specific embodiments thereof as dis- 
closed and illustrated herein are not to be considered in a 
limiting sense as numerous variations are possible. The 60 
subject matter of the inventions includes all novel and 
non-obvious combinations and subcombinations of the vari- 
ous elements, features, functions and/or properties disclosed 
herein. No single feature, function, element or property of 
the disclosed embodiments is essential to all of the disclosed 65 
inventions. Similarly, where the claims recite "a" or "a first" 
element or the equivalent thereof, such claims should be 



understood to include incorporation of one or more such 
elements, neither requiring nor excluding two or more such 
elements. 

It is believed that the following claims particularly point 
out certain combinations and subcombinations that are 
directed to one of the disclosed inventions and are novel and 
non-obvious. Inventions embodied in other combinations 
and subcombinations of features, functions, elements and/or 
properties may be claimed through amendment of the 
present claims or presentation of new claims in this or a 
related application. Such amended or new claims, whether 
they are directed to a different invention or directed to the 
same invention, whether different, broader, narrower or 
equal in scope to the original claims, are also regarded as 
included within the subject matter of the inventions of the 
present disclosure. 

We claim: 

1. A system for dynamically allocating bandwidth among 
a plurality of distributed client computers interconnected by 
a network link, comprising: 

a plurality of agents, each agent being associated with one 

of the plurality of distributed client computers; and 
a control module, wherein 

the agents are adapted to repeatedly communicate 
bandwidth allocation data for the distributed client 
computers to the control module; 
in response to the bandwidth allocation data commu- 
nicated by the agents, the control module is adapted 
to dynamically calculate a bandwidth allocation for 
each of the distributed client computers and com- 
municate the calculated bandwidth allocation to the 
agent associated with the respective distributed cli- 
ent computer; and 
each agent is adapted to receive the calculated band- 
width allocation for its associated distributed client 
computer from the control module and restrict band- 
width usage by the distributed client computer to the 
calculated allocation, the control module being 
adapted to dynamically allocate bandwidth by: 
apportioning available bandwidth to determine a 
configured bandwidth share for each distributed 
client computer during an upcoming time period; 
obtaining a prior bandwidth allocation for each dis- 
tributed client computer, wherein the prior band- 
width allocation is an amount of bandwidth allot- 
ted to each distributed client computer during a 
prior period; 

obtaining a utilized bandwidth for each distributed 
client computer, wherein the utilized bandwidth is 
an amount of bandwidth used by each distributed 
client computer during the prior period; and 

comparing the configured bandwidth share, prior 
bandwidth allocation and utilized bandwidth for 
each distributed client computer to determine an 
upcoming bandwidth allocation for each distrib- 
uted client computer for the upcoming period. 

2. The system of claim 1, wherein if the utilized band- 
width is less than the prior bandwidth allocation for any of 
the distributed client computers, the upcoming allocation for 
such distributed client computer is determined by reducing 
the prior bandwidth allocation for such distributed client 
computer. 

3. The system of claim 1, wherein if the prior bandwidth 
allocation is less than the configured bandwidth share for 
any of the distributed client computers, and if the utilized 
bandwidth is substantially equal to the prior bandwidth 
allocation for any such distributed client computer, the 
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upcoming allocation for such distributed client computer is 

determined by increasing the prior bandwidth allocation for 

such distributed client computer. 

4. A method of dynamically allocating bandwidth to a 

plurality of distributed client computers connected to a 

network, comprising the steps of: 
periodically obtaining bandwidth allocation data for each 

of the distributed client computers; 
dynamically calculating a bandwidth allocation for each 
of the distributed client computers based on the band- 
width allocation data; and 
controlling network access of each of the distributed 
client computers to limit network bandwidth usage of 
each of the distributed client computers to the calcu- 
lated bandwidth allocations, where such control is 
achieved through operation of an agent software mod- 
ule loaded into memory of each of the distributed client 
computers, wherein the step of dynamically calculating 
a bandwidth allocation for each of the distributed client 
computers is performed by: 

apportioning available bandwidth to determine a con- 
figured bandwidth share for each distributed client 
computer during an upcoming time period; 
obtaining a prior bandwidth allocation for each distrib- 
uted client computer, wherein the prior bandwidth 
allocation is an amount of bandwidth allotted to each 
distributed client computer during a prior period; 
obtaining a utilized bandwidth for each distributed 
client computer, wherein utilized bandwidth is an 
amount of bandwidth used by each distributed client 
computer during the prior period; and 
comparingmeconfiguredbandwidtrr share, prior band- 
width allocation and utilized bandwidth for each 
distributed client computer to determine a bandwidth 
allocation for each distributed client computer for the 
upcoming period. 
5_ The method of claim 4, wherein if the utilized band- 
width is less than the prior bandwidth allocation for any of 
the distributed client computers, the upcoming allocation for 
such distributed client computer is determined by reducing 
the prior bandwidth allocation for such distributed client 
computer. 

6. The method of claim 4, wherein if the prior bandwidth 
allocation is less than the configured bandwidth share for 
any of the distributed client computers, and if the utilized 
bandwidth is substantially equal to the prior bandwidth 
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allocation for any such distributed client computer, the 
upcoming allocation for such distributed client computer is 
determined by increasing the prior bandwidth allocation for 
such distributed client computer. 

7. A system for dynamically allocating bandwidth among 
a plurality of distributed client computers interconnected by 
a network link, comprising: 

a plurality of agents, each agent being associated with one 
of the plurality of distributed client computers and 
adapted to monitor and control its associated distrib- 
uted client computer by interfacing with a socket object 
used to opera lively couple the network link with appli- 
cation programs running on the associated distributed 
client computer; and 
a control module, wherein: 

the agents are adapted to communicate bandwidth 
usage data for the distributed client computers to the 
control module; 
the control module is adapted to dynamically calculate 
a bandwidth allocation for each of the agents in 
response to the bandwidth usage data; and 
each agent is adapted to interface with the socket object 
actively restrict bandwidth usage on the network link 
by its associated distributed client computer in accor- 
dance with the bandwidth allocation calculated for 
that agent, wherein the control module is adapted to 
dynamically allocate bandwidth by: 
apportioning available bandwidth to determine a 
configured bandwidth share for each distributed 
client computer during an upcoming time period; 
obtaining a prior bandwidth allocation for each dis- 
tributed client computer, wherein the prior band- 
width allocation is an amount of bandwidth allot- 
ted to each distributed client computer during a 
prior period; 

obtaining a utilized bandwidth for each distributed 
client computer, wherein the utilized bandwidth is 
an amount of bandwidth used by each computer 
during the prior period; and 

comparing the configured bandwidth share, prior 
bandwidth allocation and utilized bandwidth for 
each distributed client computer to determine an 
upcoming bandwidth allocation for each distrib- 
uted client computer for the upcoming period. 
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